This invention relates to image processing and, more particularly, to a method and apparatus for determining the spatial frequency response characteristics of a digital image acquisition system or device such as an electronic camera or scanner, when the input from the original scene content is uncontrolled, except that it contains reference signals of a given class (e.g., step edges) whose location and exact spatial properties are unknown.
Conventionally in the laboratory, the spatial frequency response (SFR) of a (linear) image acquisition system is defined as SFR(f)=O(f)/I(f), where O is output, I is input, and f is spatial frequency. SFR is measured by passing a known xe2x80x9creference signalxe2x80x9d through the system and calculating the ratio at each spatial frequency of concern. Typically, the known reference signal takes the form of a calibrated physical target containing step edges, sine waves or other known waveforms. The target is imaged through the system to digital form, a region of interest containing only the reference signal is extracted by a human, and finally the extracted digital region of interest is passed through a specially designed SFR estimation program which precisely locates, registers, and averages together multiple reference signal responses, eventually transforming to the frequency domain to obtain SFR(f). A method and a computer program intended for measuring the modulation transfer function (MTF) relating to electronic imaging for cameras and scanners are standardized and defined in the following International Standards Organization (ISO) documents: 1) ISO# 12233, WG18, Work Item 188, entitled Photographyxe2x80x94Electronic Still Picture Cameras; and 2) ISO# 16067-1, WG18, Work Item 214, entitled Photographyxe2x80x94Electronic Scanner for Photographic Images. The aforementioned ISO documents are incorporated by reference herein in their entirety for supplemental background information which is non-essential but helpful in appreciating the applications of the present invention.
While the aforementioned laboratory methodology may be fairly accurate for measuring SFR, they have some of the following shortcomings. First, the conventional methodologies are time consuming, expensive, and require laboratory level control over the content of the input scene. Second, the conventional methodologies do not take into account any differences in system behavior accruing between characterization time and later use of the system. For instance, SFR may change due to device age, temperature, physical abuse, accidental mis-focus, internally variable system parameters. etc. Third, the conventional methodologies cannot address local (intra-image) de-focussing effects such as those due to depth-of-field. Fourth, the measured SFRs of the conventional methodologies must be widely available in a standard format analogous to the ICC color profile format, in order to be useful to downstream image processing operations.
A xe2x80x9cspatial profilexe2x80x9d format is disclosed in U.S. patent application Ser. No. 08/709,487 filed Sep. 6, 1996 by Hultgren et al. and is hereby incorporated by reference herein in its entirety for background information. A device color profile is standardized and defined as xe2x80x9ca digital representation of the relation between device coordinates and a device-independent specification of colorxe2x80x9d in the International Color Consortium (ICC) Profile Format Specification, Version ICC. 1: 1998-08 incorporated by reference herein in its entirety for supplemental background information which is non-essential but helpful in appreciating the applications of the present invention.
In practice, carefully measured SFR""s in a standard spatial profile format are often not available. New digital cameras and scanners are introduced every month, and there may simply be too many to accurately characterize. Images needing enhancement may be received from the World Wide Web (WWW) and/or the Internet, and the system which produced them can be entirely unknown, making SFR calculation by the conventional methods impossible.
There is therefore a need in the art for an effective apparatus and method thereof for estimating the SFR characteristics of an image acquisition system, such as a scanner or camera, from its output images, when the original scene content cannot be carefully controlled as in the laboratory or conventional methods. Particular needs remain for an apparatus and a method for estimating SFR characteristics that can provide: an efficient and cost-effective method; the capability to take into account changes that accrue with later use of the image acquisition system; the capability to address depth-of-field; and the ability to avoid the constraints associated with requiring SFR characteristics that are widely available in some standard xe2x80x9cspatial profilexe2x80x9d format, in order to be usefull to downstream image processing operations.
By way of background, a disclosure directed towards a digital image processing system for estimating the frequency dependence and gray-level dependence of noise introduced by an image source device, and more particularly, for generating a spatial device profile describing the noise introduced by the source device is disclosed in U.S. patent application Ser. No. 08/996,810 by Reuman is hereby incorporated by reference herein in its entirety as background information.
The present invention is directed generally toward an image processing system device and method thereof for estimating the SFR of an image acquisition system or device from its output images, when the original scene content cannot be carefully controlled as in the laboratory method. By practicing the disclosed invention, the skilled practitioner can automatically or semi-automatically determine the SFR of an image acquisition or device, assuming that the scene contains specified reference signals (e.g., step edges) while the reference signals"" location, orientation, contrast, mean gray level, and spatial extent in the corresponding digitized image(s) may be unknown.
In one aspect, the present invention features a method of estimating spatial frequency response (SFR) of a digital image acquisition system or device from the images that the system or device produces from input of an original scene, wherein the estimating method comprising the following steps: gathering, from the acquired image or images, data related to the SFR of the digital image acquisition system or device; computing, from the gathered data, a collection of one or more SFR estimates which describe the gathered data; and generating, from the collection of SFR estimates and the gathered data, a single overall estimate of the SFR of the digital image acquisition system or device. In some embodiments, the present invention comprises a semi-automated step that includes gathering additional information, from a source exclusive from the gathered data from the acquired image or images, wherein the additional information is related to the SFR of the digital image acquisition system or device.
In a second aspect, the invention features an image processing system for estimating spatial frequency response (SFR) of a digital image acquisition system or device from the images that the digital acquisition system or device produces from input of an original scene, the image processing system comprising: a means for gathering, from the acquired image or images, data related to the SFR of the digital image acquisition system or device; a means for computing, from the gathered data, a collection of one or more SFR estimates which describe the gathered data; and a means for generating, from the collection of SFR estimates and the gathered data, a single overall estimate of the SFR of the digital image acquisition system or device. In an alternative embodiment, the present invention comprises a semi-automated means that includes a means for gathering additional information, from a source exclusive from the gathered data from the acquired image or images, wherein the additional information is related to the SFR of the digital image acquisition system or device.
An advantage of the present invention is that it can be fully automaticxe2x80x94no human intervention is necessary. It can therefore be put to use when human involvement is too costly for whatever reason. A fully automatic approach may be augmented if the internal system models are inconsistent with the actual behavior of the physical systems in question. In this case, a semi-automatic system is required whereby additional partial system knowledge such as system type is added by querying the user, the image header, or other available information.
Furthermore an advantage of the present invention is that it provides an effective apparatus and method thereof for estimating the SFR characteristics of an image acquisition system, such as a scanner or camera, from its output images, when the original scene content cannot be carefully controlled as in the laboratory method.
Moreover, the present invention SFR estimation apparatus and method thereof provide an efficient and cost-effective method for estimating the SFR of an image acquisition system or device.
Further yet, the present invention SFR estimation apparatus and method thereof provide the capability to take into account changes to SFR that accrue during use of the image acquisition system.
Still further, the present invention SFR estimation apparatus and method thereof provide the capability to address intra-image differences in SFR due to depth of field related focus effects.
Finally, the present invention SFR estimation apparatus and method thereof avoid the need for having standard SFR information widely available in some standard xe2x80x9cspatial profilexe2x80x9d format.